Device for monitoring surroundings of a vehicle

ABSTRACT

A device for monitoring surroundings of a vehicle mounted in a vehicle, captures an image of surroundings of the vehicle, extracts from the captured image an image area for a predetermined part of a desired type of object, and sets a predetermined area below the extracted image area. The device extracts candidates for the desired type of object present outside the vehicle based on the predetermined area and determines, for each of the extracted object candidates, whether the object candidate is the desired type of object. When no pattern different from the background is captured in the predetermined area, or when a pattern different from the background is captured only in one of a first and a second areas, then the object in the image area is excluded from the candidates for the desired type of object. Thus, a pedestrian, for example, can be distinguished from an artificial structure.

TECHNICAL FIELD

The present invention relates to a device which monitors surroundings ofa vehicle, and more specifically to a device capable of easily andefficiently distinguishing and detecting a desired type of object whenmonitoring surroundings of a vehicle.

BACKGROUND ART

Conventionally, there has been proposed a device which extracts avariety of objects present in surroundings of a vehicle. According to amethod described in Patent Literature 1 below, a distance from a vehicleto an object is measured on the basis of a temporal rate of change ofthe size of a local area which is set as an area including a part of theobject that moves relatively little within an image obtained by aninfrared camera mounted in a vehicle.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Patent No. 4486997

SUMMARY OF THE INVENTION Problems to be Solved

In the aforementioned method, an upper body area of a pedestrianincluding its head is set as a local area including the part of theobject whose movement is relatively small. The head of a living bodysuch as a pedestrian has a relatively high temperature and thus may bedetected easily by an imaging device such as a far-infrared camera. Thedistance to the object can be measured satisfactorily with a singleimaging device with the use of the local area.

However, there can be a heat source such as a street light having ashape similar to the head of a pedestrian in the surroundings of avehicle. An artificial structure such as the street light can beerroneously detected as a living body such as a pedestrian even when thefar-infrared camera is used as mentioned above.

Therefore, an object of the present invention is to improve the accuracyof detecting a desired type of object by decreasing the possibility oferroneously detecting things other than the desired type of object asthe desired type of object.

Solution to the Problems

According to one aspect of the present invention, a vehicle surroundingsmonitoring device includes an imaging unit which is mounted in a vehicleand captures an image of the surroundings of a vehicle, a unit whichextracts an image area for a predetermined part of a desired type ofobject in the captured image, a unit which sets a predetermined areabelow the extracted image area, an object candidate extracting unitwhich extracts, on the basis of the predetermined area, a candidate forthe desired type of object present outside the vehicle, and adetermination unit which determines whether the object candidate is thedesired type of object for each object candidate extracted. When nopatterns that differ from the background are captured in thepredetermined area, or when a pattern that differs from the backgroundis captured only in one of a first and a second areas each of which is ahalf of a predetermined area that is divided into the right and left inthe horizontal direction of the captured image, the object candidateextracting unit excludes the object(s) in the image area with thepredetermined area from the candidates for the desired type of object.

When the desired type is a pedestrian, the extracted image area for thehead of the pedestrian is sometimes in fact an artificial structurehaving a similar shape such as a street light. However, when the imagearea includes a head of a pedestrian, a body and legs below the headwould have a substantially symmetrical structure. When the image areaincludes an artificial structure such as a street light, a post or thelike on the lower side of the street light would not have a symmetricalstructure in most cases.

The present invention is made in view of such finding. That is, thepredetermined area below the extracted image area is examined to excludean object within the predetermined area from the candidates for thedesired type of object when no pattern is captured in the predeterminedarea, or when some pattern is captured only in one of the left side andthe right side of the predetermined area. As a result, an object such asa pedestrian can be distinguished from an artificial structure such as astreet light with a relatively simple operation.

The operation for determining a type of the extracted object candidate,that is, determining whether the object candidate is a human such as apedestrian or an artificial structure such as a building, generallyincludes image processing for examining a shape characteristic of anobject, and image processing for examining the behavior of the object bytracking the object in time sequence. Thus, the operation requires arelatively high operational load. The operational load would increasewhen such type determination process is performed on all the extractedobject candidates. According to the present invention, the operationalload of the type determination process can be reduced by performing thetype determination process after exclusion in the aforementioned manner.

According to an embodiment of the present invention, the imaging unit isan infrared camera whereas the extracted image area is a high-intensityarea having an intensity value higher than a predetermined value in thecaptured image. The object candidate extracting unit examines left-rightsymmetry of the object in the first and the second areas on the basis ofthe intensity value in each of the first and the second areas and, whendetermination is made that the left-right symmetry of the object is low,excludes the object in the image area from the candidates for thedesired type of object. As a result, the candidate for the desired typeof object can be extracted more accurately with a simpler operation onthe basis of the intensity value with the use of the infrared camera.

According to an embodiment of the present invention, the objectcandidate extracting unit determines that the left-right symmetry of theobject is low when a difference between the sum of the intensity valuesof pixels in the first area and the sum of the intensity values ofpixels in the second area is not smaller than a predetermined value, orwhen a difference between the variance of the intensity values of thepixels in the first area and the variance of the intensity values of thepixels in the second area is not smaller than a predetermined value.Thus, the left-right symmetry between the first and the second areas canbe determined using the intensity values, whereby the candidate for thedesired type of object can be extracted more accurately with the simpleroperation.

The other features and advantages of the present invention will becomeapparent from the detailed description to be given hereinafter.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a vehiclesurroundings monitoring device according to an embodiment of the presentinvention.

FIG. 2 is a diagram for illustrating a mounting position of a cameraaccording to an embodiment of the present invention.

FIGS. 3( a) to 3(c) are diagrams illustrating a set-up embodiment of alower area according to an embodiment of the present invention.

FIGS. 4( a) and 4(b) are diagrams illustrating another set-up embodimentof the lower area according to an embodiment of the present invention.

FIG. 5 is a flowchart of an object determination process performed by animage processing unit according to an embodiment of the presentinvention.

FIGS. 6( a) and 6(b) are diagrams schematically illustrating a presentimage and a past image according to an embodiment of the presentinvention.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will now be described withreference to the drawings. FIG. 1 is a block diagram illustrating aconfiguration of a device which monitors the surroundings of a vehicleaccording to an embodiment of the present invention. The device includesan infrared camera 1 which is mounted in a vehicle and can detect afar-infrared ray, an image processing unit 2 which detects an object inthe surroundings of the vehicle on the basis of the image data capturedby the camera 1 and determines a type of the object, a speaker 3 whichprovides an alarm by sound (speech) on the basis of the result of thedetermination, and a head-up display (hereinafter referred to as an HUD)4 which displays the image captured by the imaging with the camera 1 anddisplays the alarm on the basis of the result of the determination. Thesurroundings monitoring device further includes a yaw rate sensor 6which detects a yaw rate of the vehicle, and a vehicular velocity sensor7 which detects a traveling speed (vehicular velocity) of a vehicle. Adetection result by these sensors is sent to the image processing unit 2and is used for a predetermined image processing as needed.

In the present embodiment, as illustrated in FIG. 2, the camera 1 isdisposed at a front part of the vehicle 10 on a central axis thereofrunning through the center of the vehicle width in order to capture animage in front of the vehicle 10. The infrared camera 1 has acharacteristic that the higher the temperature of an object is, thehigher the level of an output signal of the camera is (that is, theintensity in a captured image would increase).

The image processing unit 2 includes an A/D conversion circuit whichconverts an input analog signal into a digital signal, an image memorywhich stores a digitized image signal, a central operation processingunit (CPU) which performs various operation process, a RAM (randomaccess memory) used by the CPU at the time of operation to store data, aROM (read-only memory) which stores a program performed by the CPU aswell as data (including a table and a map) used by the CPU, and anoutput circuit which outputs a drive signal for the speaker 3 and adisplay signal for the HUD 4, for example. The output signal from thecamera 1 is converted into a digital signal and is input to the CPU. Asillustrated in FIG. 2, the HUD 4 is provided on a front window of thevehicle 10 such that a screen 4 a is displayed at a position in front ofa driver, whereby the driver can visually recognize the screen displayedon the HUD 4.

In lieu of HUD 4, a display unit may be mounted to a dashboard. Forexample, a display unit of a so-called navigation device (not shown) canbe used as the display unit. As is widely known, the navigation deviceis a device which can detect a current position of a vehicle, calculatean optimum route to a destination, and display the current position andthe route on map information.

FIGS. 3( a) to 3(c) illustrate the basic idea of the present invention.FIG. 3( a) is a diagram schematically illustrating a pedestrian capturedby a camera, while FIGS. 3( b) and 3(c) are diagrams schematicallyillustrating a street light captured by a camera. A head of thepedestrian indicated by a shaded area in FIG. 3( a) is often an exposedpart of a human body and thus can be extracted relatively stably as ahigh-temperature object, that is, an image area with high intensity. Onthe other hand, as indicated by shaded areas in FIGS. 3( b) and 3(c),the street light is a heat source and, therefore, is highly possiblyextracted as the high-temperature object, that is, the image area withhigh intensity.

Thus, in the present embodiment, the desired type of object is set to apedestrian, and an image area for a head of the pedestrian is extracted.A high-intensity area with an intensity value higher than apredetermined value can be extracted as the image area for the head ofthe pedestrian by using the infrared camera. As mentioned above,however, there can be a case where the extracted image area (thehigh-intensity area in the present embodiment) indicates not the head ofthe pedestrian but an artificial structure such as a street light.

Therefore, in the present invention, a predetermined lower area 31 isset below the extracted high-intensity area. The lower area 31 is setsymmetrically about a reference line 33 serving as a center andextending in a vertical direction along the center of gravity or thecenter of the extracted high-intensity area. That is, the lower area 31is divided into two parts, a left side (first) area 31L and a right side(second) area 31R.

The lower-body below the head (a torso and legs) of a pedestrian has asubstantially symmetrical structure. Therefore, as illustrated in FIG.3( a), when the high-intensity area indicated by the shaded arearepresents the head of the pedestrian, it is expected that some patternincluding the torso of the pedestrian (some object different from thebackground) is captured in both of the left side area 31L and the rightside area 31R of the lower area 31 below the head.

On the other hand, a column (post) part 35 below a heat source part (theshaded part) of the street light as illustrated in FIGS. 3( b) and 3(c)does not have a symmetrical structure about the reference line 33,whereby a pattern is captured only in one of the left side area 31L andthe right side area 31R as illustrated in FIG. 3( b) or is capturedneither in the left side area 31L nor the right side area 31R asillustrated in FIG. 3(c). In the latter case, it is only the backgroundpart such as the sky which is captured in the lower area 31.

Accordingly, in the present invention, the desired type of object (thepedestrian in the present embodiment) is detected distinguishably fromthe object other than the desired type of object (the artificialstructure such as the street light in the present embodiment) on thebasis of the presence of a pattern or the left-right symmetry thereof inthe lower area 31 set below the high-intensity area.

As illustrated in the figure, the pedestrian has a substantiallysymmetrical structure about the reference line 33 from the torso to thelegs, whereas the post 35 of the street light down to the ground doesnot have a symmetrical structure. As a result, the lower area 31 set inthe aforementioned manner needs only be below the shaded high-intensityarea and not be in contact therewith. As illustrated in FIG. 4( a), forexample, an area below the high-intensity area down to where it contactsthe ground may be set as the lower area 31. In this case, a foot of thepedestrian or a point of contact between the post of the street lightand the ground is extracted as an edge, whereby an area in the verticaldirection from the edge to the high-intensity area can be set as thelower area 31. Alternatively, as illustrated in FIG. 4( b), an area incontact with the ground away from the high-order area may be set as thelower area 31.

Moreover, the lengths of the lower area 31 in a horizontal(left-and-right) direction and a vertical (up-and-down) direction can beset to an arbitrarily predetermined appropriate value. The length of thelower area in the horizontal direction is set greater than the width ofthe body of the pedestrian in FIGS. 3( a) to 4(b) but may also be setshorter than the width of the torso, in which case the left-rightsymmetry of the torso of the pedestrian would likewise be exhibited inthe lower area 31.

FIG. 5 is a flowchart illustrating a process performed by the imageprocessing unit 2 according to an embodiment of the present invention.The process is performed at a predetermined time interval. In thepresent embodiment, as illustrated in FIGS. 3( a) to 3(c), it is assumedthat the lower area is set in contact with the area extracted as thehigh-intensity area. Moreover, as described above, it is assumed thatthe desired type of object is the pedestrian.

In step S11, an output signal from the far-infrared camera 1 (namely,data of a captured image) is received as an input, undergoes A/Dconversion, and is stored in the image memory. The stored image data isa gray scale image having a higher intensity value as the temperature ofthe object is higher.

In step S12, the gray scale image undergoes a binarization process.Specifically, an area brighter than an intensity threshold ITH isdetermined to be “1” (white) while an area darker than the intensitythreshold is determined to be “0” (black). The intensity threshold ITHcan be determined by an arbitrary and appropriate method. An object suchas a heat source of a living body or an artificial structure which has atemperature higher than a predetermined temperature is extracted as awhite area (high-intensity area) by the binarization process.

In step S13, the binarized image data is converted into run length data.Specifically, in the area that has become the white area by thebinarization, the run length data is represented by coordinates of astarting point (the left end pixel of the white area in each pixel line)of the white area in each pixel line (hereinafter referred to as a line)and the length (expressed in a pixel number) from the starting point toan end point (the right end pixel in each line). Here, the y axis istaken in the vertical direction of the image, and the x axis is taken inthe horizontal direction of the image. For example, when the white areain a pixel line with the y coordinate of y1 is a line from (x1, y1) to(x3, y1), this line formed of three pixels is expressed by the runlength data of (x1, y1, 3).

In step S14, the object is labeled so that a candidate for the desiredtype of object is extracted. That is, among lines in the form of the runlength data, lines having sections that lie on top of one another in they direction are collectively considered as one object, to which a labelis given. As a result, one or a plurality of object candidates isextracted (detected).

The detected object candidate has temperature higher than apredetermined temperature, thereby possibly including not only a livingbody such as a human (pedestrian) and an animal but also an artificialstructure such as a street light having a heat source. FIG. 6( a)schematically illustrates an embodiment of the image (referred to as apresent image) obtained by the process performed this time. In thisembodiment, a head 101 of the pedestrian as well as a street light 102are extracted as the high-temperature objects (namely the high-intensityareas) as indicated by the shaded areas.

In step S15, a tracking process is performed on the object to determinewhether or not the object candidate detected this time is identical toan object candidate detected when the process was performed in the past.If the cycle of performing the process is shown by k, for example,whether or not the object candidate detected in step S14 last time (timek−1) is identical to the object candidate detected in step S14 this time(time k) is determined. This determination can be performed by anarbitrary method. For example, as described in JP 2001-6096 A, theobject at time k and the object at time k−1 can be determined identicalwhen the difference in coordinates of the center of gravity, in areas,or in aspect ratios of a circumscribed square of the objects is notlarger than a predetermined tolerance between the both objects.Alternatively, by using the gray scale image, the identity determinationmay be performed on the basis of correlation (similarity) between anintensity distribution of the object candidate detected this time withinthe image area and an intensity distribution of the object detected inthe past within the image area candidate. For example, the objects canbe determined identical when the difference in the variances of theintensity between the objects is not larger than a predetermined value,indicating that the correlation is high.

FIG. 6( b) schematically illustrates an embodiment of the image(referred to as a past image) obtained prior to FIG. 6( a) (the lasttime in this embodiment). The head 101 of the pedestrian as well as thestreet light 102 are extracted as the high-temperature objects(high-intensity areas), namely the object candidates, as indicated bythe shaded areas, and are determined to be identical to the objects 101and 102 illustrated in FIG. 6( a).

In step S16, the lower area 31 described above is set below thehigh-intensity areas 101 and 102 extracted this time. The lower area 31is set symmetrical about the reference line 33 extending in the verticaldirection in the image through the center of gravity or the center ofthe detected high-intensity area, namely, the detected high-temperatureobject. The lower area 31 thus includes the left side area 31L and theright side area 31R.

The lower area 31 is set to have a predetermined size in the presentembodiment but may also be set to have the size that varies inaccordance with the size of the high-intensity area. For example, thelength of the lower area may be set no shorter than the longest part ofthe high-intensity area in the horizontal direction and set at apredetermined length in the vertical direction.

In step S17, the lower area 31 set in the present image illustrated inFIG. 6( a) for the object candidate determined to be identical in stepS16 is used as a mask to search for an area with the highest similarity(highly correlated) to the lower area in the past image illustrated inFIG. 6( b). For example, in the case of the high-intensity area 101 inFIGS. 6( a) and 6(b), the lower area 31 set in FIG. 6( a) is used as themask, which is superposed and moved downward over the high-intensityarea 101 of the image in FIG. 6( b) to search for the area with thehighest similarity to the mask. As described above, the area where thedifference between the variance of the intensity of the pixels in themask and the variance of the intensity in the superposed image is thesmallest can be searched, and the variance, the difference in which isthe smallest between the mask and the superposed image, can be used asan index to indicate the similarity. An area 131 found as a result ofthe search is illustrated in the past image in FIG. 6( b). The similarprocess is performed for a high-intensity area 102.

Although not illustrated in FIG. 5, it is preferred, when there is a gap(difference) of not smaller than a predetermined value in the horizontaldirection between a line diving the searched area 131 in half so thatthe divided areas become symmetric and the reference line 33 set on thebasis of the high-intensity area 101 in the past image, the object inthe high-intensity area is excluded from the pedestrian candidates byproceeding to step S24 without proceeding to step S18. The large “gap”implies, for example, that an object at a different distance is capturedoverlapping the object in the high-intensity area in either the presentimage or the past image, thereby degrading the accuracy of calculationof the position (distance) of the pedestrian.

In step S18, it is determined whether some pattern is present in thelower area 31 set in the present image. This step examines the presenceof some pattern different from the background in the lower area 31 belowthe high-intensity area. When the object is the pedestrian having atorso and legs beneath the head, a pattern as described above should bepresent. Specifically, a pattern different from the background isdetermined present when the calculated variance of the intensity valueof the pixels in the lower area 31 is not smaller than a predeterminedvalue, and that no pattern different from the background is determinedto be present if the variance is less than the predetermined value. Itmay also be determined that the pattern is “not a pedestrian” (such as autility pole) when the pattern is present (there is some object) only inone of the right and the left areas of the lower area 31 dividedthereinto.

When the object is a street light illustrated in FIG. 3( c), forexample, there is no pattern but the background such as the sky capturedin the lower area 31, whereby the variance of the intensity would below. On the other hand, when the object is a pedestrian illustrated inFIGS. 3( a), 6(a) and 6(b), a torso is captured (imaged) in the lowerarea 31, whereby the variance of the intensity would be high. Whendetermination is made that no pattern is present, the object in thehigh-intensity area accompanied with the lower area 31 is excluded fromthe pedestrian candidate in step S24. The object excluded from thepedestrian candidate would not be subjected to a subsequent typedetermination process (step S26). In the embodiment illustrated in FIG.6( a), the body of the pedestrian and the post of the street light arecaptured (imaged) in the lower area 31 of the high-intensity area 101and the lower area 31 of the high-intensity area 102, respectively,whereby it is determined Yes in step S18 to proceed to step S19.

In step S19, the lower area 31 set in the present image is compared tothe lower area 131 set in the past image as the result of the search instep S17.

In step S20, as a result of the comparison, determination is madewhether the similarity between the lower areas is greater than or equalto a predetermined value. The variance of the intensity can be used asthe index that indicates the similarity, as described above. Therefore,upon calculating the variance of the intensity value of the pixels inthe lower area 31 in the present image as well as the variance of theintensity value of the pixels in the lower area 131 in the past image,the similarity (correlation) between the lower areas is determined highwhen the difference in the variances is less than or equal to thepredetermined value, and the process proceeds to step S21. Thesimilarity between the two is determined low when the difference in thevariances is greater than the predetermined value, in which case theprocess proceeds to step S23.

In step S21, the lower area 31 in the present image is divided into twoparts, namely the right and the left sides, along the reference line 33serving as the center. In step S22, it is determined whether a patterndifferent from the background is present in both of the left side area31L and the right side area 31R of the lower area 31. It is expectedthat, when the object is the pedestrian, some pattern different from thebackground would be captured in both of the left side area 31L and theright side area 31R as described above because the body is captured inthe lower area 31. When the object is the street light, on the otherhand, a pattern would be captured only in one of the left side area 31Land the right side area 31R, if any, as illustrated in the figure. Thisstep thus examines the presence of such pattern in each lower area 31.

Specifically, the variance of the intensity value of the pixels in eachof the left side area 31L and the right side area 31R is calculated toexamine whether the difference between the variances is greater than orequal to the predetermined value. When the difference is greater than orequal to the predetermined value, the pattern such as the post of thestreet light is only captured in one of the areas, whereby determinationis made that the left-right symmetry between the left side area 31L andthe right side area 31R is low. In this case, the process proceeds tostep S24 where the object in the high-intensity area accompanied withthe lower area 31 is excluded from the pedestrian candidate. On theother hand, when the difference between the variances is less than thepredetermined value, some pattern is captured in both of the left sidearea 31L and the right side area 31R, whereby determination is made thatthe left-right symmetry between the areas is high. In this case, theprocess proceeds to step S25 where the object is determined to be thepedestrian candidate.

Alternatively, the sum of the intensity values may be used in place ofthe variance of the intensity values. The sum of the intensity values ofthe pixels in the left side area 31L and the sum of the intensity valuesof the pixels in the right side area 31R are calculated to examinewhether the difference between the sums is greater than or equal to apredetermined value. When the difference is greater than or equal to thepredetermined value, the pattern is captured only in one of the areas,whereby determination is made that the left-right symmetry is low, andthe process proceeds to step S24. When the difference is less than thepredetermined value, some pattern is captured in the both areas, wherebydetermination is made that the left-right symmetry is high, in whichcase the process proceeds to step S25. Note that, in determining thesymmetry, the sum of absolute difference (SAD) may be used in place ofor in addition to the variance or the sum of the intensity values.

When the similarity between the lower area 31 in the present image andthe lower area 131 in the past image is not greater than or equal to thepredetermined value back in step S20, the process proceeds to step S23where it is examined whether the pedestrian has any uniquecharacteristic. The unique characteristic of the pedestrian can beextracted by an arbitrary method that is appropriate from a viewpoint ofa shape or a walk cycle, for example. It can be examined, for example,whether a shape characteristic considered as legs (such as two portionsextending in the vertical direction) is extracted from the image areathat is extracted, the image area covering from the high-intensity areato the ground, or whether a cyclic characteristic based on the walk isextracted by tracking the image area covering from the high-intensityarea to the ground from the past to the present (refer to JP 2007-264778A, for example). The width of the extracted image area in the horizontaldirection can be set to the length of the longest part in thehigh-intensity area plus a predetermined margin, for example. When nounique characteristic of the pedestrian is extracted, the processproceeds to step S24 where the object is excluded from the pedestriancandidate, whereas when the characteristic is extracted, the processproceeds to step S25 where the object is determined to be the pedestriancandidate.

In step S26, a type determination process is performed to determinewhether each pedestrian candidate determined in step S25 is ultimately apedestrian. It can be specified more accurately whether or not thepedestrian candidate is a pedestrian by this type determination process.

An arbitrary and appropriate method can be used to perform the typedetermination process. For example, a known pattern matching can be usedto determine whether or not the object candidate is a pedestrian byexamining the characteristic regarding the shape of the object candidateor examining the behavior or the like of the object candidate throughtime. The type determination process for a pedestrian can employ amethod described in, for example, JP 2007-241740 A, JP 2007-264778 A, JP2007-334751 A, and the like. In addition to the pedestriandetermination, an artificial structure determination and an animaldetermination may be performed in the type determination process. Theobject candidate that has not been determined to be a pedestrian canthus be determined whether it is an artificial structure or an animal.In this case, the type determination process for an artificial structurecan employ a method described in, for example, JP 2003-016429 A, JP2008-276787 A, and the like. Moreover, the type determination processfor an animal can employ a method described in, for example, JP2007-310705 A, JP 2007-310706 A, and the like.

Moreover, a distance of the object determined to be a pedestrian by thetype determination may be calculated by an arbitrary method that isappropriate. For example, as illustrated in Japanese Patent No. 4486997,by setting a local area including a high-intensity area (which is set asan area with relatively little movement when the object is a pedestrian)and finding a rate of change of the size of the local area in the pastimage to the local area in the present image, the distance of the objectcan be calculated on the basis of the rate of change and a vehicularvelocity detected by the vehicular velocity sensor 7. As a result, theposition of the object relative to an own vehicle can be specified.

In step S27, an alarm related to the object determined to be apedestrian is given to a driver. For example, the alarm may inform thedriver of the distance calculated as described above or may go off whenthe distance value becomes a predetermined value or less. As illustratedin JP 2001-6096 A, for example, the alarm may also go off whendetermination is made that there is a high possibility for the object tocome in contact with the vehicle by examining the possibility of thecontact on the basis of the distance value. The alarm may be given byunit of a sound through the speaker 3 or of a visual display through theHUD 4.

Therefore, in the present invention, the artificial structure such asthe street light is excluded from the pedestrian candidate when thedesired type is the pedestrian by examining whether some patterndifferent from the background is present in the lower area below theimage area (the high-intensity area in the present embodiment) for thehead of the pedestrian, or whether some pattern different from thebackground is present only in one of the right and the left areas of thelower area divided thereinto. The accuracy of extracting the pedestriancandidate can thus be improved by the relatively simple operation.Moreover, the exclusion process has a role to liberally (roughly) filterthe extracted object candidates prior to the type determination processand operates to exclude the object candidate less likely to be thedesired type from the subsequent type determination process. As aresult, the object candidate that can be the noise in the typedetermination process can be excluded in advance. The operational loadof the type determination process can be decreased by the exclusionprocess performed prior to the type determination process. Furthermore,there is only one camera required to be mounted in a vehicle, therebyallowing the cost to be decreased.

The lower area 31 is set in contact with the high-intensity area in theaforementioned flow but may also be set to cover the area from thehigh-intensity area to the ground or set below the high-intensity areaat a distance therefrom as described with reference to FIGS. 4( a) and4(b). In this case, for example in step S16, the portion where the footor the post of the street light comes in contact with the ground wouldappear as an edge in the captured image, so that the edge is extractedby searching below the high-intensity area. The lower area 31 may be setto the portion covering from the extracted edge to the lower end of thehigh-intensity area (the case in FIG. 4( a)) or to the area having apredetermined length above the extracted edge (the case in FIG. 4( b)).

The present embodiment has illustrated a case where the head isextracted as the high-intensity area when the image of the pedestrian iscaptured. The pedestrian can be detected distinguishably from theartificial structure such as the street light having the heat sourcewhich has a similar shape to that of the head of the pedestrian, evenwhen the head of the pedestrian alone is extracted as the high-intensityarea. The similar process can be performed when not only the head butthe body and the legs of the pedestrian are extracted as thehigh-intensity area. For example, within the extracted high-intensityarea, the portion determined to be a substantially circular area isdetermined as the head area so that the lower area can be set below thehead area.

In the aforementioned embodiment, the image area for the head of thepedestrian can be extracted relatively easily on the basis of theintensity value by using the infrared camera. Alternatively, however,another camera such as a visible camera may be used. In this case, theimage area for the head of the pedestrian (such as a circular imagearea) is likewise extracted by an arbitrary known method so that thelower area may be set below the image area. By using a pixel value inthe lower area, it may be determined whether or not a pattern differentfrom the background is captured in the lower area, or whether or not thepattern different from the background is captured in both of the leftand the right side areas of the lower area divided thereinto.

Moreover, the aforementioned embodiment has illustrated that thepedestrian candidate can be extracted with more superior accuracy byexcluding the artificial structure such as the street light even whenthe similarity of the objects examined between the present image and thepast image is high. It is not always required to examine the similarityof the objects through time, however. In another embodiment, thepedestrian candidate may be determined by setting the lower area in thepresent image and examining the lower area as described above withoutexamining the similarity. In this case, the operational load can befurther reduced because the object needs not be tracked through time.

While the embodiments of the present invention have been describedabove, the present invention is not limited to these embodiments but maybe modified for use without departing from the spirit of the presentinvention.

REFERENCE SIGNS LIST

-   1 Infrared camera (imaging unit)-   2 Image processing unit-   3 Speaker-   4 HUD

The invention claimed is:
 1. A vehicle surroundings monitoring devicecomprising: an imaging unit which is mounted in a vehicle and capturesan image of surroundings of the vehicle; and an image processing unithaving a processor and a memory, the image processing unit beingconfigured to: extract possible image areas for a predetermined part ofa desired type of object in the captured image; set a predetermined areabelow each extracted image area; when no pattern different from abackground is captured in the predetermined area, or when a patterndifferent from the background is captured only in one of a first and asecond areas each of which is a half of the predetermined area dividedin a horizontal direction, exclude the extracted area related to saidpredetermined area and determine candidates for the desired type ofobject as objects in the remaining extracted image areas; determine, foreach of the candidates, whether the candidate is the desired type ofobject.
 2. The device according to claim 1, wherein the imaging unit isan infrared camera, wherein the image processing unit is configured toextract the possible image areas as high-intensity areas each having anintensity value higher than a predetermined value, and wherein the imageprocessing unit is configured to determine whether left-right symmetryof an object in the first and the second areas is lower than apredetermined level by examining the left-right symmetry on the basis ofintensity values in the first and the second areas, and to exclude theextracted image region related to said first and second areas if thedetermination is positive.
 3. The device according to claim 2, whereinthe image processing unit is configured to determine that the left-rightsymmetry is lower than a predetermined level when a difference between asum of intensity values of pixels in the first area and a sum ofintensity values of pixels in the second area is not smaller than apredetermined value, or when a difference between a variance of theintensity values of the pixels in the first area and a variance of theintensity values of the pixels in the second area is not smaller than apredetermined value.
 4. A method of monitoring surroundings of avehicle, comprising steps of: capturing an image of surroundings of thevehicle with an imaging unit; extracting possible image areas for apredetermined part of a desired type of object in the captured image;setting a predetermined area below each extracted image areas; when nopattern different from a background is captured in the predeterminedarea, or when a pattern different from the background is captured onlyin one of a first and a second areas each of which is a half of thepredetermined area divided in a horizontal direction, excluding theextracted area related to said predetermined area and determinecandidates for the desired type of objects as objects in the remainingextracted image areas; determining, for each of the candidates, whetherthe candidate is the desired type of object.
 5. The method according toclaim 4, wherein the step of extracting comprises extracting thepossible image areas for a predetermined part of a desired type ofobject as high-intensity areas each having an intensity value higherthan a predetermined value, and wherein the step of excluding comprises:determining whether left-right symmetry of an object in the first andthe second areas is lower than a predetermined level by examining theleft-right symmetry on the basis of intensity values in the first andthe second areas, and excluding the extracted image region related tosaid first and second areas if the determination is positive.
 6. Themethod according to claim 5, wherein said excluding an object in theimage area comprises: determining that the left-right symmetry is lowerthan a predetermined level when a difference between a sum of intensityvalues of pixels in the first area and a sum of intensity values ofpixels in the second area is not smaller than a predetermined value, orwhen a difference between a variance of the intensity values of thepixels in the first area and a variance of the intensity values of thepixels in the second area is not smaller than a predetermined value.